Literate programming
University of Luxembourg
Thursday, the 14th of November, 2024
Quarto: an open-source scientific and technical publishing system
Literate programming is the practice of mixing code and descriptive writing in order to execute and explain a data analysis simultaneously in the same document.
— William Landau Developer of {targets}
For a bioinformatician, pure programming is barely happening, we need reports/websites where content and formatting are disconnected
Even better to separate content and rendering, outsourced to continuous integration/development.
Source: {targets} user manual
Predecessor of Quarto.
Version 1 in 2016, the idea was to have text and code in one document, execute code and let the final markdown converted to different formats by pandoc.
Jupyter notebook were also exploring the same concept. Rendering order led sometimes to some frustration
Credit: Artwork by Allison Horst
<!DOCTYPE html>
<html>
<body>
<h1>This is a heading</h1>
<p>This is some text in a <b>paragraph</b></p>
<h2>This is a second level heading</h2>
<ul>
<li><a href="http://exa.com"><code>site</code></a>
<li><img src="https://images.computerhistory.org/revonline/images/500004391-03-01.jpg?w=200">
</ul>
</body>
</html>Extended Rmarkdown. Languages and Integrated Development Environment agnostic
.qmd) document to various formats (PDF, HTML…)knitrYAML header
Free text in Markdown
Code chunks
Code to be interpreted
knitr for jupyter for QuartoNotebookRunner.jl package for juliaObservabledocx)How to add equations
Enclose in $ for in-line equations
$a^2+b^2=c^2$ renders as \(a^2+b^2=c^2\).
Double ($$) for separate equations.
$$G_{\mu v}=8 \pi G (T_{\mu v} + \rho _\Lambda \ g_{\mu v}) $$
renders as:
\[G_{\mu v}=8 \pi G (T_{\mu v} + \rho _\Lambda \ g_{\mu v})\]
_quarto.yml, at project rootquarto render_quarto.yml
project:
type: website
render:
- "*.qmd"
- "!resources/"
- "!data/"
execute:
freeze: auto # re-render only when source changes
website:
title: "bulk RNA-seq"
site-url: https://misb.pages.uni.lu/rna-seq
image: img/visualization.png
description: "Snakemake dependency graph"
search:
location: navbar
type: overlay
navbar:
background: light
logo: img/visualization.png
left:
- file: index.qmd
text: ""
- file: slides.qmd
text: " Lectures"
- file: practicals.qmd
text: " Practicals"
- file: setup.qmd
text: " Setup tutorial"
- file: exams.qmd
text: " Exams"
right:
- file: about.qmd
text: " About"
page-footer:
left: |
All content licensed under a
except for _source:_ statements.
center: "Copyright © [Université du Luxembourg](https://www.uni.lu/) 2024. All rights reserved."
right:
- icon: facebook
href: https://www.facebook.com/uni.lu
- icon: linkedin
href: https://www.linkedin.com/school/166650
- icon: instagram
href: https://www.instagram.com/uni.lu
- icon: youtube
href: https://www.youtube.com/user/luxuni
border: true
format:
html:
page-layout: article
code-block-border-left: true
code-block-bg: true
code-copy: true
code-link: false # requires pkg downlit
theme:
light: cosmo
dark: [cosmo, theme-dark.scss]
fontsize: 1.2em
css: styles.css
toc: true
lang: en
date-format: long
highlight-style: atom-oneWith previous _quarto.yml and some CSS tweaks. Grid listing
shiny apps (using CI scheduled pipelines)Command to add an extensions among many available (+165 references)
dlsm-bioinfo/
├── bioinfo.qmd
├── dag-atac-seq.png
├── dag-rna-seq.png
├── insert-citation.png
├── orbi-bib-export.png
├── _extensions
│ └── quarto-journals
│ └── plos/
├── gaigneaux.bib
├── ginolhac.bib
├── references.bib
├── plos2015.bst <- bib style
├── _manuscript/ <- outputs
└── _quarto.yml <- main configDemo HTML output
PLoS PDF example:
Compiled here
git , as you do anyway right?Some icons from flaticon.com, credit to Valueinvestor
.github/workflows/publish.yml
on:
workflow_dispatch:
push:
branches: main
name: Quarto Publish
permissions:
contents: write
pages: write
jobs:
build-deploy:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4
- name: Set up Quarto
uses: quarto-dev/quarto-actions/setup@v2
with:
version: pre-release
- uses: r-lib/actions/setup-r@v2
- uses: r-lib/actions/setup-r-dependencies@v2
- name: Publish to GitHub Pages (and render)
uses: quarto-dev/quarto-actions/publish@v2
with:
path: quarto-intro.qmd
target: gh-pages
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}.gitlab-ci.yml
image: alpine:latest
stages:
- build_image
- build_site
- deploy
build docker image:
stage: build_image
image: docker:20.10.12-dind
before_script:
- docker info
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
script:
- 'docker pull $CI_REGISTRY_IMAGE:latest || true'
- docker build --cache-from $CI_REGISTRY_IMAGE:latest --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --tag $CI_REGISTRY_IMAGE:latest .
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
- docker push $CI_REGISTRY_IMAGE:latest
only:
changes:
- Dockerfile
build_site:
stage: build_site
image: $CI_REGISTRY_IMAGE:latest
cache:
key: $CI_JOB_NAME
paths:
- _freeze/
- _site/
artifacts:
name: "$CI_JOB_NAME"
expire_in: 2 days
paths:
- _site/
before_script:
# https://mastodon.social/@eddelbuettel/111693310455494722
- |
Rscript -e "read.dcf('DESCRIPTION', 'Imports') |>
tools:::.split_dependencies() |>
names() |>
setdiff(tools:::.get_standard_package_names()$base) |>
install.packages()"
script:
- quarto render
when: always
Deploy to BASV53:
stage: deploy
only:
- main
dependencies:
- build_site
before_script:
- 'which ssh-agent || ( apk add --update openssh )'
- eval $(ssh-agent -s)
- 'which rsync || ( apk add --update rsync )'
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh && chmod 700 ~/.ssh
- 'echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- rsync -azh -e "ssh -p 8022" --delete --filter 'protect /preview' $CI_PROJECT_DIR/_site/ $SSH_HOST:/var/www/basv53/Dockerfile, thus 3 steps.Dockerfile
FROM rocker/r2u:22.04
RUN apt-get update && \
apt-get install -y \
curl netbase \
zip git \
libxml2-dev libcurl4-openssl-dev libmagick++-dev \
&& rm -rf /var/lib/apt/lists/*
RUN install.r tidyverse rmarkdown biobase knitr V8 gt gtextras ggimage ggrepel ggupset BiocManager && \
installBioc.r deseq2 apeglm
# from https://quarto.org/docs/get-started/
ARG QUARTO_VERSION="1.6.33"
RUN curl -LO https://github.com/quarto-dev/quarto-cli/releases/download/v${QUARTO_VERSION}/quarto-${QUARTO_VERSION}-linux-amd64.deb && \
apt-get update -qq && apt-get -y install \
./quarto-${QUARTO_VERSION}-linux-amd64.deb && rm quarto-${QUARTO_VERSION}-linux-amd64.deb \
&& apt autoremove -y && apt clean -y && rm -rf /var/lib/apt/lists/*r2u is awesome, offering packages as APT.
{tidyverse} install is < 10 secTyler Morgan-Wall, author of rayshader
Summary
Quarto (qmd)Markdownrender to different output formatsFurther reading 📚
Acknowledgments 🙏 👏
Thank you for your attention!